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Introduction 


At every stage since the start of the pandemic, decisions across education and childcare 
have been informed by the scientific and medical evidence — both on the risks of 
coronavirus (COVID-19) infection, transmission and illness, and on the known risks to 
children and young people not attending education settings — balancing public health and 
education considerations. 


Children and young people are at very low risk of serious illness from COVID-19 
infection’? and preliminary UK Health Security Agency (UKHSA) analysis estimated a 
lower risk of hospitalisation among Omicron cases in school-aged children (5- to 17-year- 
olds) compared to Delta cases in the same age group.* 


The Government's priority is for all children and young people to continue to be able to 
attend schools, colleges and early years settings, and for those settings to be able to 
deliver face-to-face, high-quality education. The evidence is clear that missed face-to- 
face attendance causes significant harm to children and young people’s education, life 
chances and mental and physical health. This harm disproportionately affects children 
and young people from the most disadvantaged backgrounds.® 


This summary sets out the evidence informing the Government's decision to revisit the 
guidance on the use of face coverings within secondary schools and colleges in England 
— temporarily extending their recommended use in communal areas to also include 
classrooms and teaching spaces for those in year 7 and above. This decision has been 
taken on the recommendation of UKHSA and is based on a range of evidence. In making 
this decision, the Government has balanced education and public health considerations, 
including the benefits in managing infection and transmission, against any educational 
and wider health and wellbeing impacts from the recommended use of face coverings. 


Evidence shows that face coverings can contribute to reducing transmission of COVID- 
19 primarily by reducing the emission of virus-carrying particles when worn by an infected 
person. The direct COVID-19 health risks to children and young people are very low. 


1 PHE, JCVI advice on COVID-19 vaccination of children aged 12 to 15 (3 September 2021) 

2 SAGE: Children’s Task and Finish Group: Update to 17 December 2020 paper on children, schools and 
transmission (10 February 2021) 

3 Ward et al., Risk factors for PICU admission and death among children and young people hospitalized 
with COVID-19 and PIMS-TS in England during the first pandemic year (20 December 2021) 

4 UKHSA, Investigation of SARS-CoV-2 variants: technical briefings - GOV.UK (www.gov.uk 
December 2021) 

5 University College London, Academic paper prepared for SAGE, Impacts of school closures on the 
physical and mental health of children and young people: a systematic review (2 July 2021) 

6 DfE, Evidence summary: COVID-19 - children, young people and education settings - GOV.UK 
(www.gov.uk) (July 2021) 
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Vaccination reduces the risk of infection and onward transmission, as well as reducing 
the risk of severe outcomes for children, young people and adults. However, at this point 
in time, high infection rates do present a challenge to maintaining face-to-face education 
for pupils, students and staff. Reducing transmission in education settings in turn 
supports the Government’s priority of maximising attendance and minimising disruption 
to in-person learning, particularly as pupils, students and staff return to settings this term 
and supports Plan B measures to reduce transmission in wider society. 


The government will continue to evaluate the data relating to all COVID-19 measures, 
including in education settings. The temporary guidance on face coverings will be 
reviewed alongside the review of Plan B measures. The measures in place day-to-day in 
education and childcare settings are based on the latest public health advice. We will 
continue to work with UKHSA to carefully monitor the data and the evidence to continue 
to strike a balance between managing transmission risk and reducing disruption to 
children and young people’s education. 


Impact of face coverings on transmission of COVID-19 


Face coverings can be effective in contributing to reducing transmission of COVID-19 in 
public and community settings. This is informed by a range of research, including 
randomised control trials, contact tracing studies, and observational studies — assessed 
most recently by UKHSA, described in a review conducted in November 2021.’ The 
review’s conclusions were broadly in line with those of a previous Public Health England 
review; however, the addition of randomised control trials and substantially more 
individual-level observational studies increases the strength of the conclusions and 
strengthens the evidence for the effectiveness of face coverings in reducing the spread of 
COVID-19 in the community, through source control, wearer protection, and universal 
masking. 


Person-to-person transmission of COVID-19 can occur by direct transmission of droplets 
(respiratory particles with ballistic trajectory that directly deposit on mucous membranes 
such as the lining of the mouth and nose), by airborne transmission of aerosols (smaller 
respiratory particles that remain suspended in the air and can be inhaled), or by touching 
the eyes, nose or mouth after direct contact with surfaces on which these virus-carrying 
particles have deposited. The effectiveness of face coverings stems mostly from reducing 
the emission of these virus-carrying particles when worn by an infected person (Source 
control). They may provide a small amount of protection to an uninfected wearer; 
however, this is not their primary intended purpose. 


While the evidence to date is from earlier variants, SAGE has advised that there are 
preliminary indications that Omicron might show more airborne transmission; this would 
make the use of face coverings, alongside mitigations such as ventilation, even more 
important than for Delta.®° In a paper assessing the use of non-pharmaceutical 
interventions in the context of Omicron, they advised that “wearing face coverings in as 
many indoor environments as is practicable will help to reduce transmission, and that in 
the current circumstances it may be necessary to reconsider the wearing of face 
coverings in places where the balance of risks and benefits did not previously support it, 
for example primary school classrooms....”'° Although government guidance remains 
that children under 11 should be exempt from requirements to wear face coverings, the 


7 UKHSA: The effectiveness of face coverings to reduce transmission of COVID-19 in community settings. 
(November 2020). The review includes 25 studies (including 9 preprints and 2 non-peer and descriptive 
studies (not included in this review) and assessed evidence for the efficacy of face coverings (not 
addressed in this review). In total, 3 studies (all contact tracing studies) in the previous review were also 
used in this review. 


8 SAGE: SAGE 98 minutes: Coronavirus (COVID-19) (7 December 2021) 


° EMG and SPI-B: Non-Pharmaceutical Interventions in the context of Omicron (15 December 2021) 
10 EMG and SPI-B: Non-Pharmaceutical Interventions in the context of Omicron (15 December 2021) 
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balance of risks for secondary classrooms has changed at this point in time, in 
accordance with the evolving evidence and the phase of the pandemic. 


Face coverings (worn correctly and of suitable quality) are likely to be most effective at 
reducing transmission in settings when people are likely to be close together.'’ Face 
coverings can reduce the risk of transmission when people are in close proximity or small 
spaces even for short durations of time. They cannot compensate for poor ventilation but 
may further reduce risks of longer-range airborne transmission when people are in 
shared air for longer periods of time, and this effect may be more important in poorly 
ventilated spaces.'* The virus can remain airborne for many hours in closed indoor 
spaces such as classrooms.'*'4 As face coverings can reduce both aerosols and 
droplets, they are potentially beneficial for mitigating both close range transmission (less 
than 2m distance) and aerosol transmission, particularly in poorly ventilated indoor 
environments.’ Other proportionate measures will remain in place in education settings 
to help reduce the transmission of the virus, such as increased ventilation, good hygiene 
and testing. 


Wearing face coverings is comparatively cheap and easy to implement and supervise. It 
can be a visible outward signal of safety behaviour and a reminder of COVID-19 risks. 
Different types of coverings (cloth masks, surgical masks, N95 masks etc) have varying 
levels of effectiveness at capturing particles, but all can be useful if worn correctly."® 
Three-layered cloth masks can have a comparable filtration performance to surgical 
masks so long as the face seal is adequate enough to minimize leakage.” 


Face masks and coverings will become highly contaminated with upper respiratory tract 
and skin micro-organisms. Disposal of single-use face coverings could theoretically pose 
a risk of transmission for inappropriately discarded face coverings, but it is very likely that 
the reduction in transmission risk due to reduced droplet and aerosol emissions from 
wearing a face covering significantly outweighs any potential for enhanced risk of 
transmission through inadvertent contact with a contaminated face covering. This is likely 
to hold regardless of duration that the face covering is used. Risks associated with 


11 SAGE EMG Application of physical distancing and fabric face coverings in mitigating the B117 variant 
SARS-CoV-2 virus in public, workplace and community settings (13 January 2021) 

12 SPI-B, SPI-M and EMG: Considerations for potential impact of Plan B measures (13 October 2021) 
13 Spitzer, Masked education? The benefits and burdens of wearing face masks in schools during the 
current Corona pandemic (2020) 

14 NERVTAG/EMG: SARS-CoV-2: Transmission Routes and Environments (October 2020) 

15 SAGE: NERVTAG/EMG: Duration of wearing of face coverings (15 September 2020) 

16 UKHSA: The role of face coverings in mitigating the transmission of SARS-CoV-2 (October 2021) 


17 Robinson et al, Efficacy of face coverings in reducing transmission of COVID-19: Calculations based on 
models of droplet capture (2021) 


contaminated face coverings can be mitigated by hand hygiene, surface cleaning, and 
proper disposal or proper washing. "8 


18 SAGE: NERVTAG/EMG: Duration of wearing of face coverings (15 September 2020) 
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Impact of face coverings in education settings 


Whilst not conclusive, there are now a number of scientific studies which consider the 
association between COVID-19 and the use of face coverings specifically in education 
settings. The review of evidence conducted by UKHSA included evidence from studies in 
schools and summer camp settings.1? These were observational and therefore due to 
limitations in the studies, the results provide less direct evidence of the effectiveness of 
face coverings than randomised control trials. The results were mixed but taken together 
support the conclusion that the use of face coverings in schools can contribute to 
reducing COVID-19 transmission. There have also been several studies in the USA, 
comparing schools in US counties with and without mask requirements for students, 
including two recent studies from the US Centres for Disease Control and Prevention.?°: 
21 These were excluded from the latest UKHSA review (due to the type of study design) 
or published after the cut-off date, but generally find higher rates of COVID-19 in schools 
without mask requirements, compared to those with mask requirements. There are also 
modelling studies looking at the potential impact of face coverings — for example, a study 
assessing impacts of the use of face coverings in schools and society in September- 
October 2020, which suggest that mandating face coverings in secondary schools, in 
addition to other parts of society, could reduce the number of infections.*4 


DfE has also undertaken initial observational analysis based on data reported by 123 
secondary schools that implemented face coverings during a 2-3-week period in the 
autumn term 2021, compared to a sample of similar schools that did not (see Annex A). 
The preliminary findings demonstrate a potential positive effect in reducing pupil absence 
due to COVID-19. The analysis has yet to be peer reviewed — DfE intends to undertake 
further work to validate and strengthen it, with a view to bringing it closer to the approach 
taken for existing evidence on face coverings, such as the studies considered in 
UKHSA’s review. 


A survey conducted by the Department for Education in March 2021 found that pupils 
had a somewhat positive attitude towards wearing face coverings. Pupils generally 
agreed that face coverings made others (87%) and themselves (70%) feel safe. 
However, 80% of pupils reported that wearing a face covering made it difficult to 
communicate, and more than half felt wearing one made learning more difficult (55%).?° 


19 UKHSA: The effectiveness of face coverings to reduce transmission of COVID-19 in community settings. 
(November 2020). 


20 Budzyn et al., Paediatric COVID-19 Cases in Counties with and Without School Mask Requirements — 
United States, (2021) 

21Jehn et al., Association Between K-12 School Mask Policies and School-Associated COVID-19 
Outbreaks — Maricopa and Pima Counties, United States (2021) 

22 Panovska-Griffith et al., Modelling the potential impact of mask use in schools and society on COVID-19 
control in the UK (2021) 


23 DfE, Parent and pupil panel: omnibus surveys — GOV.UK (www.gov.uk) (March 2021) 
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In a Unison survey of support staff, 71% said face coverings in secondary school 
classrooms are an important safety measure.”4 


Wearing face coverings may have physical side effects and impair face identification, 
verbal and non-verbal communication between teacher and learner.” This means there 
are downsides to face coverings for pupils and students, including detrimental impacts on 
communication in the classroom. At different times during the pandemic, the risks of 
potential negative impacts on teaching, learning and wider health and wellbeing have 
needed to be balanced against the benefits of face coverings — making this a balanced 
judgement. 


A survey conducted by the Department for Education in April 2021 found that almost all 
secondary leaders and teachers (94%) thought that wearing face coverings has made 
communication between teachers and students more difficult, with 59% saying it has 
made it a lot more difficult.2° 


Research into the effect of mask wearing on communication has found that concealing a 
speaker's lips led to lower performance, lower confidence scores, and increased 
perceived effort on the part of the listener. Moreover, meta-cognitive monitoring was 
worse when listening in these conditions compared with listening to an unmasked 
talker.2” A survey of impacts on communication with mask wearing in adults reported that 
face coverings negatively impact hearing, understanding, engagement, and feelings of 
connection with the speaker.2® People with hearing loss were impacted more than those 
without hearing loss. The inability to see facial expressions and to read lips have a major 
impact on speech understanding for those with hearing impairments.?° The worse the 
hearing, the greater the impact of the mask.°° 


The WHO reports that “the wearing of masks by children with hearing loss or auditory 
problems may present learning barriers and further challenges, exacerbated by the need 
to adhere to the recommended physical distancing. These children may miss learning 
opportunities because of the degraded speech signal stemming from mask wearing, the 


24 UNISON, Face coverings in schools make staff feel safer (March 2021) 

25 Spitzer, Masked education? The benefits and burdens of wearing face masks in schools during the 
current Corona pandemic (2020) 

2 DFE, School snapshot panel: COVID-19 - GOV.UK (www.gov.uk) (April 2021) 

7 Giovanelli et al., Unmasking the Difficulty of Listening to Talkers with Masks: lessons from the COVID-19 
pandemic (2021) 

28 Saunders et al., Impacts of face coverings on communication: an indirect impact of COVID-19 (2021) 
29 National Deaf Children’s Society, Face coverings in education — National Deaf Children’s Society 
position paper (2021) 

30 Homans & Vroegop, The impact of face masks on the communication of adults with hearing loss during 
COVID-19 in a clinical setting (2021) 


elimination of lipreading and speaker expressions and physical distancing”.*" It is 
therefore important to continue to offer flexibility to education settings either not to use 
face coverings or to use transparent face coverings to meet the needs of pupils with 
hearing impairments. The Department for Health and Social Care has been working on a 
range of initiatives to enable the creation and availability of safe and effective transparent 
masks including piloting transparent face masks in a range of settings, to further explore 
their usability.°? 


Government guidance on the use of face coverings makes clear that there are 
circumstances where people may not be able to wear a face covering — for example due 
to illness, impairment or disability, or where communication relies on lip reading, clear 
sound or facial expressions — and advises education settings to be mindful and respectful 
of such circumstances. 


The potential difficulty of wearing face coverings correctly is likely to be exacerbated for 
younger children, who find it harder to adhere to recommendations on hygienic use and 
feel the impact on communication more acutely than older children. Throughout the 
pandemic, evidence continues to confirm that children can be susceptible to COVID-19 
infection although a range of analyses suggest that in the absence of vaccination, 
children are less susceptible to infection than adults. The evidence is stronger for pre- 
school and primary-aged children. 


Government guidance continues to be that children aged under 11 years old should be 
exempt from requirements to wear face coverings in all settings including education. 
UKHSA does not recommend face coverings for children under the age of 3 years for 
health and safety reasons.*° There are a range of measures to mitigate transmission in 
primary schools and early years settings which include staff testing, staff face 
coverings in communal areas, daily testing for close contacts and improved ventilation. 


31 WHO: Advice on the use of masks for children in the community in the context of COVID-19 (August 
2020) 


32 DHSC: DHSC recognises the importance of transparent face masks to make communication easier (July 
2021) 


33 DHSC: Face coverings: when to wear one, exemptions, and how to make your own (December 2021) 
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Annex A — preliminary DfE analysis on the use of face 
coverings in secondary schools 


This analysis is based on available data from schools reporting. It is preliminary, 
experimental analysis, which would benefit from robust external peer review to a longer 
timescale. 


Results 


In a weighted sample of secondary schools that did not use face masks, the average 
COVID-19 absence rate* fell by 1.7 percentage points from 5.3% on 1 October 2021 to 
3.6% in the 3rd week of October. This is equivalent to a 32% decrease. 


In secondary schools that did use face coverings? (either face coverings only or a 
combination of face masks and additional communications e.g. providing more 
communications to parents but not introducing any further measures such as increased 
testing), the average COVID-19 absence rate fell by 2.3 percentage points from 5.3% on 
1 October 2021 to 3.0% in the 3 week of October. This is equivalent to a 43% decrease. 


At surface level, this suggests that COVID-19 absence fell by 0.6 percentage points more 
(an 11% relative difference) in secondary schools that used face masks compared to 
similar schools that did not over a 2—3-week period. 


There is a level of statistical uncertainty around the result. The analysis is non-peer 
reviewed and with the current sample size, shows a non-statistical and unknown clinical 
significant reduction in infection in a short follow up period, including that a ‘false positive’ 
(i.e. finding that face coverings saw reduced absence when the finding is actually by 
chance) would emerge around 15% of the time; a 5% threshold is widely used to declare 
statistical significance in academic literature. 


Therefore, further work should be done to extend the analysis in terms of scope: for 
example, looking at different statistical methodologies, capturing different and longer 


34 For the purposes of this analysis COVID-19 related absence is being used as an outcome (which can 
help to quantify the amount of the learning lost due to pupils being ill with COVID-19), rather than as a way 
to understand transmission of COVID-19 within the community. Transmission of COVID-19 within the 
community is dependent on a number of factors (including but not limited to: age profiles of people infected 
at different times in the outbreak, transmission rates between different age groups, school, catchment 
areas and fluctuating levels of testing); these factors are outside the scope of the model, which is intended 
to give an indication of the association between the use of face masks within schools and COVID-19 
absence rates (which are an indication of lost learning). 


35 The use of face coverings is self-reported as an additional control measure by schools in the Educational 
Settings form. The data does not differentiate between face coverings being used in communal areas or 
classrooms or both. 
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treatment time periods and controlling for a wider number of school and local area 
variables to ensure this is a consistent finding. 


Methodology 


The most reliable way to measure the impact of the use face coverings would be to runa 
randomised-controlled trial (RCT) comparing the change in the COVID-19 absence rate 
between schools randomly assigned to be in the treatment group (i.e., using face 
coverings) versus schools assigned to the control group (i.e., not using face coverings) 
over time but, due to the speed and ever-changing nature of the pandemic, conducting 
an RCT is not possible. However, DfE has been collecting administrative data about face 
coverings directly from schools via the Educational Settings form. Using this data, we can 
observe COVID-19 absence rates in schools that used face coverings, but it should be 
noted that we cannot assess causality, i.e., whether the use of face coverings resulted in 
X% absence rate. 


Using data from the Educational Settings form, COVID-19 absence rates were calculated 
across the latter part of the first half term of AY2021/22. For this analysis, COVID-19 
absence rate is defined as the proportion of pupils who were absent from a setting due to 
suspected or confirmed COVID-19 cases. Note this differs from published COVID-19 
absence statistics which include other reasons, such as absence due to attendance 
restrictions. 


Schools that continuously implemented face coverings (as an additional measure either 
in isolation or alongside strengthened communications only, e.g. providing more 
communications to parents, but not introducing any further measures such as increased 
testing) for at least two weeks were identified. This included additional use in classrooms 
and/or communal areas without differentiating between these. Not every school returned 
this form every day, so if a school recorded that they were using face coverings in 
consecutive forms that were returned days apart, it was assumed that they were also 
implementing face masks on the missing school days. A period of two weeks was used 
because it is longer than the 10-day isolation period for those who test positive for 
COVID-19. From these schools, those that started using face coverings for the first time 
on 1 October 2021, and who had COVID-19 absence rate data in the third week from 
implementing face coverings and in the week prior to doing so were selected as the 
‘treatment’ group. This date was specified because it maximised the sample size when 
viewing face coverings as a point treatment, i.e., this was the day on which the largest 
proportion of schools fitting the criteria started using face coverings. The size of the 
treatment group for this analysis was 123 schools. 


A control group was constructed by taking schools that did not use any additional 
measures in the first half term and that had filled out the form on 01/10/2021, the week 
prior and in the third week afterwards. Exploration of the data showed that the control 
and treatment group had differing characteristics, so weights for the control group 
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schools were calculated using entropy balancing. More detail on the impact of the 
weighting can be found in Chart 1 and Chart 2 below. The characteristics balanced on 
were: headcount (grouped into categories), proportion of pupils eligible for free school 
meals, proportion of pupils of a minority ethnic background, COVID-19 absence rate on 1 
October 2021 and averaged COVID-19 absence rate in the week prior. A total of 1,192 
schools were included in the control group. Average COVID-19 absence rates for the 
face covering group and weighted control group on 1 October 2021 and 2-3 weeks later 
were calculated and these are provided in Table 1. Averaging over a third week allowed 
us to include more schools in the sample, rather than requiring a school to have 
submitted a response on a specific date in that week. 


A difference-in-differences analysis was calculated to compare the change in COVID-19 
absence rates between the two groups. This was conducted via a logistic regression of 

treatment (face coverings or no face coverings) and time (1 October 2021 or two weeks 
later) against COVID-19 absence rate. The resulting coefficient, p value and confidence 
intervals are shown in Table 2. 


Caveats 


This only uses data from the first half term of 2021/22 so does not include any Omicron 
data. 


This analysis does not take into account differential usage of face masks in different 
settings. The data from the Educational Settings form does not differentiate between 
whether face coverings were used in classrooms or communal areas. We compared 
schools with similar initial COVID-19 absence rates as one way of trying to account for 
the possibility that schools implementing face coverings may have done so due to their 
COVID-19 absence rates being higher than usual. 


This analysis does not account for other potential causes of differences in absence rates 
(Local Authority wider practice, measures outside of the school’s control, etc). 


Absence rate on 1 October 2021 and average absence rate in the 3rd week of October 
are analysed as “point treatments”; any differences in the length of time face masks were 
used prior to 1 October 2021 were not taken into account. 


These results are for schools starting to use these measures from 1 October 2021; the 
relationship could have been different at different points in the term. 


To minimise the effect of any single school on the sample average, a weighting threshold 
of 50 was used. The highest weight given to a control group school was 32, with the five 
highest weighted control group schools having a combined weight of 77. This should be 
noted but is not considered too high for this analysis. Using a different maximum 
weighting threshold could result in slightly different results. 
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Next steps 


Including additional variables in the regression model may help to account for other 
factors that could be contributing to the association between the use of face coverings 
and COVID-19 absence. Additional variables we would like to explore using in future 
models include, for example: community COVID-19 case rates, regional data (LA, 
information on LA wider response to COVID-19, etc), other characteristics of pupils 
(proportion of pupils with SEND, etc) and any information on differential use of face 
coverings. 


The current analysis uses 1 October 2021 and the third week of October 2021 as a fixed 
‘point treatment’ and ‘post-treatment period’. Further analysis would explore the impact of 
using different dates, using a range of ‘start dates’ and changing the length of treatment 
time. Panel data methods or marginal structural models could allow for different 
treatment lengths to be considered in the analysis (which could potentially allow further 
insight into the impact of using face coverings for varying lengths of time). 


Exploring alternative regression specifications to better fit the data may yield more 
precise effect estimates and offer reassurance on the consistency of the finding when 
using a variety of statistical methods. 


The current analysis was also carried out using a two-tailed test, assuming face mask 
usage could result in a decrease or increase in COVID-19 absence. It may be useful to 
revise to a one-tailed test (i.e., only testing for a decrease) to assess whether this 
increases the significance of the current finding that face coverings saw reduced absence 
due to COVID-19. 


This set of further collective analyses would benefit from independent replication of 
results by external peer review. DfE is working on making the underlying data, coding, 
and statistical analysis available to an external peer reviewer for independent replication 
of results. 
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Charts and tables 


Table 1: descriptive statistics and COVID-19 absence rates 


Weighted 
Schools using control group 
face coverings schools 


Number of schools 123 1192 


Proportion of schools* 3.6% 35.0% 


Initial average COVID-19 absence 
rate 5.3% 5.3% 


Average COVID-19 absence rate 2- 
3 weeks later 3.0% 3.6% 


*Number of secondary schools taken from Get Information About Schools database 


Table 2: regression results 


Coefficient -0.202 
Significance (p) value 0.148 
Lower Cl of coefficient (2.5%) -0.476 
Upper Cl of coefficient (97.5%) 0.072 


Chart 1 below demonstrates the impact of applying weights (generated from entropy 
balancing) on the shape of the data. 


Prior to weighting, the schools in the treatment group have a wider spread of COVID-19 
absence rates compared to the control group. After weighting, the spread of COVID-19 
absence rates in the control group is more similar to that of the treatment group, creating 
a more like-for-like comparison for the difference-in-differences analysis. 


The wider spread of data in the treatment group prior to weighting is likely partially due to 
the smaller sample size, as variation between individual schools is more visible in a 
smaller sample. Sample size may also explain the greater statistical uncertainty 
associated with the significance value of 15%. A greater sample of schools in the 
treatment group would likely give a more precise estimate, however, this is not possible 
due to the limited number of schools using face mask policies simultaneously. Further 
work could explore expanding the time-period under study to potentially yield more 
precise estimates. 
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% absence related to COVID-19 in secondary schools, by treatment group and time period 
Density plots 


Unweighted data comparison 
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Chart 1: Impact of applying weights on the shape of the data. 


Chart 2 below demonstrates the impact of applying weights (generated from entropy 
balancing) on the mean and quantiles of COVID-19 absence rates in the control and 
treatment groups. The ‘box’ extends from the 25t percentile to the 75" percentile, with 
the ‘whiskers’ or lines extending to the furthest value at most 1.5 times the interquartile 
range. Values beyond the end of the whiskers are plotted individually as outliers. Note 
that in general boxplots show the median; here, the dotted line shows the mean, for 
consistency with the headline statistics. For reference, the numerical values of the means 
represented in the chart are also provided in Table 3. 


Prior to weighting, the mean absence rate of the control group increases across the 
treatment period, whereas the mean absence rate of the treatment group decreases. 
However, the absence rates in the control group remain lower overall than those in the 


treatment group. 
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% absence related to COVID-19 in secondary schools, by treatment group and time period 


Unweighted data comparison 
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Chart 2: Impact of applying weights on the mean and quantiles of COVID-19 absence rates 


After applying the weighting, the initial COVID-19 absence rate for the control group 
increases to match the initial rate of the treatment group. This allows us to assess what 
the change in absence in the control group would likely have been had the control group 
started with the same initial absence rate as the treatment group; if both groups were to 
start at the same initial absence rate, it is likely the absence rate in both groups would 
decrease across the treatment period, but the absence rate would decrease more for the 
treatment group. 


Table 3: Average COVID-19 absence rate before and after weighting 


Average COVID-19 absence (%) 


15t October 15" — 215' October 
(average) 

Unweighted 
Treatment group 5.3 3.0 
Control group 2.8 2.9 

Weighted 

Treatment group 5.3 3.0 
Control group 5.3 3.6 
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Appendix 1 — Additional information on Entropy 
Balancing 


Entropy balancing was first mentioned by Hainmueller (Massachusetts Institute of 
Technology) in 2012. It is a method for matching treatment and control observations by 
constructing a set of matching weights that force certain balance metrics to hold. It is 
considered to be a generalisation of propensity score weighting techniques. In effect, it 
makes the control group more comparable to the treatment group. 


The rationale for choosing entropy balancing as the matching method for this analysis 
was that it provided a mechanism for balancing the characteristics of treatment and 
control groups, thus reducing the risk of regression to the mean and allowing for a more 
like-for-like comparison using difference in differences post-matching. It enables us to 
keep greater numbers of cases in the analysis too — which is important when 
observational study numbers are low. 


“Entropy balancing provides a generalization of the propensity score weighting 
approach... [but] entropy balancing obviates the need for continual balance checking and 
iterative searching over propensity score models that may stochastically balance the 
prespecified covariates. Entropy balancing relies on a maximum entropy reweighting 
scheme that calibrates unit weights so that the reweighted treatment and control group 
satisfy a potentially large set of prespecified balance conditions that incorporate 
information about known sample moments. Entropy balancing thereby exactly adjusts 
inequalities in representation with respect to the first, second, and possibly higher 
moments of the covariate distributions. These balance improvements can reduce model 
dependence for the subsequent estimation of treatment effects. The method assures that 
balance improves on all covariate moments included in the reweighting.”°° 


36 Hainmueller, (2012) Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce 
Balanced Samples in Observational Studies 
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